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ABSTRACT 


A  general  purpose  nonlinear  programming  method  and 
computer  code  are  presented.  The  method  is  basically  heuristic 
but  ^extremely  simple  and  reliable.  Interactive  operation  of  the 
code  and  its  performance  on  several  test  problems  is  described. 
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BACKGROUND 


Quite  commonly,  problems  in  the  statistical  and  engineer¬ 
ing  sciences  require  the  optimization  of  some  function  for  their 
solution.  Such  problems  include  those  of  resource  allocation, 
portfolio  selection,  curve  and  surface  fitting  (e.g. ,  linear 
or  nonlinear  regression),  approximation  theory,  signal  processing 
algorithms  (optimizing  a  model  fit) ,  and  optimal  control  (via 
discretization).  In  addition,  systems  of  nonlinear  equations  can 
be  solved  by  minimizing  some  function  of  the  magnitudes  of  the 
residuals.  As  for  the  statistical  sciences  in  general  we  have 
the  following  recent  comment  of  C.  R.  Rao;  "All  statistical  pro¬ 
cedures  are,  in  the  ultimate  analysis,  solutions  to  suitably 
formulated  optimization  problems"  [I], 

Throughout  this  report  we  shall  be  concerned  with 
optimizations  of  the  particular  form: 

minimize  f(x),  x  e  n,  (1) 

where  n  is  a  solid  subset  of  some  finite  dimensional  Euclidean 
space,  describable  by  finitely  many  contraints.  More  precisely, 
we  assume  that  n  can  be  expressed  as 

n  =  |xeR”:  a^£x^<b^,  l£i<,n,  Cj<gj(x)£dj,  l£jim|,  (2) 

with  the  understanding  the  Jl  so  defined  has  a  non-empty  interior 
(is  "solid").  No  qualitative  assumptions  on  the  objective 
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function  f  in  (1)  or  on  the  constraint  functions  in  (2),  such 
as  convexity,  dif ferentiabi3.ity,  etc. ,  are  made. 

Problems  of  the  form  of  (1)  are  known  as  mathematical 
programs.  They  have  been  the  subject  of  an  immense  amount  of 
study  over  the  last  forty  years,  beginning  with  the  well-known 
special  case  of  linear  programs,  wherein  the  constraint  and 
objective  functions  are  linear  functions  of  their  variables.  As 
a  result  a  considerable  body  of  both  theory  and  computational 
algorithms  has  been  developed,  usually  under  convexity  and/or 
differentiability  assumptions.  The  books  [2-8]  provide  a 
representative  description  of  these  developments;  in  addition, 
there  are  specialized  journals  such  as  Journal  of  Optimization 
Theory  and  Applications  (since  1967)  and  Mathematical  Programming 
(since  1970). 

There  is  a  basic  dichotomy  in  programming  algorithms: 
they  may  be  designed  to  converge  to  local  or  global  minima. 

Recall  that  a  local  minimum  of  f  is  a  point  such  that 

f(x)^f(xQ)  for  every  x  in  (2  that  is  sufficiently  near  to  x^, 
while  a  global  minimum  is  indeed  a  solution  of  (1) .  Local  minima 
can  be  characterized  or  at  least  partially  identified  in  various 
ways,  depending  on  the  nature  of  the  problem.  A  prototypical 
necessary  condition  for  x^  to  be  a  local  minimum  of  a  differentiable 
progreim  (subject  to  a  "constraint  qualification"  on  the  geometry 


of  is  due  to  Kuhn  and  Tucker  [9]  and  might  be  considered  bo  be  the 
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fundamental  theorem  of  mathematical  programming.  In  its  geometric 
form  it  states  in  essence  that  the  negative  gradient  of  f  at  Xq 
must  belong  to  the  cone  generated  by  the  gradients  at  of  the 
active  constraints  there.  Further,  under  various  generalized 
convexity  hypotheses,  e.g.,  that  f  should  be  pseudoconvex  and  the 
g^  quasiconvex,  any  local  minimum  is  actually  a  global  minimum; 
in  such  a  case,  then,  the  Kuhn-Tucker  condition  characterizes  the 
solutions  of  (1).  This  remark  applies  a  fortiori  to  the  case 
where  all  functions  are  actually  convex. 

Such  characterizations  are  sometimes  used  to  provide 
stopping  criteria  for  numerical  algorithms.  For  example,  should 
a  local  minimum  occur  at  an  interior  point  of  f? ,  as  would  certainly 
be  the  case  if  there  were  no  constraints,  the  K-T  condition 
reduces  to  the  vanishing  of  the  gradient  Vf  of  f  there,  and  in 
practice  an  algorithm  might  be  terminated  at  a  point  where  1  jVf  |  | 
becomes  sufficiently  small.  Of  course,  such  a  point  will  not 
necessarily  be  a  local  minimum  nor  even  close  to  one  in  the  absence 
of  convexity  conditions.  Similarly  the  popular  "method  of  feasible 
directions"  [3,10]  attempts  to  terminate  at  a  Kuhn-Tucker  point. 

In  the  important  special  case  where  all  constraint 
functions  are  linear,  so  that  n  is  a  simplex,  the  constraint 
qualification  is  automatically  satisfied.  In  this  case  several 
effective  algorithms  exist  provided  that  V f  is  available.  These 
include  the  method  of  feasible  directions,  the  gradient  projection 
algorithm  [11],  and  an  approximation  method  wherein  f  is  replaced 
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by  its  first  order  Taylor  expansiai;  this  approximation  yields  a  linear  program 
and  the  algorithm  generates  a  sequer^oe  of  such  programs  [12] . 

In  this  linearly  constrained  case  there  are  very 
efficient  finite  algorithms  available  provided  the  objective 
function  f  is  either  linear  or  else  convex  and  quadratic. 

Failing  this,  but  with  Vf  available,  the  various  methods  mentioned 
in  the  preceding  paragraph  generate  sequences  of  linear  programs, 
systems  of  linear  equations,  and  line  searches  (one-dimensional 
optimizations)  to  be  solved.  Yet  other  methods  generate  sequences 
of  quadratic  programs  to  be  solved  by  constructing  quadratic 
approximations  to  f  [13,  14].  When  Vf  is  not  available  but 
can  be  assumed  to  exist  and  be  smooth,  a  powerful  but  rather  in¬ 
volved  algorithm  has  recently  been  described  in  [15]. 

For  the  general  (nonlinear ly)  constrained  mathematical 
program  there  are  adaptations  of  the  above  algorithms  obtained 
by  linearizing  the  contraints.  There  is  also  a  variety  of 
indirect  methods  which  may  operate  outside  the  feasible  set  fi. 
Examples  are  cutting  plane  algorithms  and  the  use  of  penalty 
functions  [2,  4].  The  former  generate  a  sequence  of  linear 
programs  over  a  decreasing  sequence  of  simplices  which  enclose 
the  possiblity  of  doing  so  depends  on  a  standard  trick  of 
utilizing  a  linear  objective  function  at  the  cost  of  introducing 
an  additional  variable  and  constraint.  The  latter  are  functions 
that  are  large  off  n  and  which  are  added  to  f ;  a  sequence  of  these 
new  c*jective  functions  is  then  minimized  without  constraints.  All  these 


methods  involve  assorted  numerical  difficulties  as  well  as  the 
possiblity  of  termination  outside  fi.  Should  the  objective 
function  be  undefined  or  meaningless  off  this  outcome  is 
naturally  untenable. 

In  the  last  decade  there  has  been  upsurge  of  interest 
in  the  global  optimization  problem  per  se;  see  [16,  17,  18]. 

This  problem  in  full  generality  is  very  different  from  and  more 
difficult  than  the  problem  of  local  optimization  just  discussed. 
There  is  relatively  little  theory  in  support  of  proposed  methods 
unless  fairly  strong  restrictions  are  placed  on  the  problems 
to  which  they  are  applied. 

The  methods  may  roughly  be  classified  as  deterministic 
or  probabilistic,  although  many  methods  have  both  features.  Most 
of  the  effort  has  gone  into  unconstrained  optimization  or  else 
into  problems  which  only  involve  bounds  on  the  variables. 

Of  these  the  probabilistic  methods  appear  in  general 
more  attractive,  in  part  because  of  the  lack  of  hypotheses  that 
must  be  imposed  on  the  problem  structure  and  in  part  because 
they  seem  less  sensitive  to  increases  in  dimension.  Another 
difference  concerns  the  manner  of  validation  of  particular 
algorithms;  with  probabilistic  methods  it  may  be  possible  to 
give  a  theoretical  a  priori  measure  of  tne  probability  of  "success", 
while  for  a  deterministic  algorithm  such  a  probability  can  at 
best  be  estimated  only  through  extensive  testing. 
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Among  the  several  popular  probabilistic  methods  we 
mention  specifically  only  two;  the  multistart  method  and  the 
sample  function  method.  The  former  is  essentially  an  iterative 
cycle  of  random  searches  throughout  Q  followed  by  application  of 
a  local  minimization  algorithm.  The  latter  hypothesizes  that  the 
objective  function  f  is  a  sample  function  of  a  known  stochastic 
process  on  The  statistics  of  the  process  then  permit  new 

points  in  to  be  generated  according  to  the  information  provided 
by  the  values  of  f  at  preceding  points.  Further  statistical 
testing  can  be  employed  to  assess  the  likelihood  that  f  could 
indeed  be  a  sample  path  of  such  a  process.  Usually  the  Wiener 
process  is  assumed.  At  this  date  the  theory  and  computational 
requirements  seem  to  limit  this  method  to  small  (perhaps  only  11) 
dimensional  problems. 

To  conclude  this  brief  background  we  note  that  a  general 
summary  of  computer  codes  for  mathematical  programming  that  have 
been  tested,  documented,  and  are  available  to  the  public  occurs 
in  119].  Somewhat  earlier  there  appeared  a  collection  of  FORTRAN 
listings  of  optimization  codes,  along  with  brief  descriptions  of 
the  algorithms  and  their  operations  [20].  The  potential  user 
is  left  to  make  his  own  choice  as  to  which  method  will  best  serve 
his  purpose  and,  unfortunately,  the  various  codes  are  not  of 
uniformly  high  quality. 
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II. 


METHODOLOGICAL  AND  PROGRAMMING  DESIDERATA 


The  authors'  experience  with  numerous  "real-world" 
optimization  problems  has  persuaded  them  of  the  need  for  a  simple 
reliable  optimization  routine  that  could  be  interactively  operated. 
Such  a  program  should  in  particular  be  applicable  to  an  essentially 
arbitrary  objective  function,  about  which  little  or  no  prior 
information  may  be  available,  save  a  procedure  for  its  evaluation. 
This  situation  occurs,  for  example  when  the  objective  function 
represents  the  output  of  some  "black  box"  operation,  perhaps 
from  a  process  optimization  or  simulation  model.  In  any  event 
the  nature  of  such  a  function  cannot  be  well  understood,  gradients 
are  unavailable  except  via  finite  difference  approximations, 
and  convexity  (occasionally  even  continuity)  is  in  doubt.  It  is 
also  often  the  case  that  the  feasible  region  must  be  strictly 
respected,  in  that  nonfeasible  points  have  no  significance  for 
the  problem. 

We  feel  that  the  nonavailability  of  analytic  gradients 
precludes  the  use  of  the  various  local  search  methods  mentioned 
in  Section  I.  While  some  codes  have  been  developed  that  utilize 
difference  approximations,  this  is  in  general  well  known  to  be  a 
numerically  risky  and  often  unstable  procedure.  There  are  the 
twin  perils  of  too  large  a  differencing  interval  (leading  to 
truncation  errors)  and  too  small  an  interval  (leading  to  round¬ 
off  and  cancellation  errors) .  The  situation  is  not  hopeless 


[15,  21,  22]  but  it  is  subtle  and  one  we  think  worth  avoiding, 
if  possible. 

The  requirements  of  simplicity,  robustness,  and  generality 
lead  us  to  a  combination  of  global  random  search  and  local  pattern 
search.  This  last  term  describes  a  class  of  methods  wherein 
the  objective  function  is  systematically  evaluated  at  several  points 
in  a  neighborhood  of  a  current  point,  the  resulting  function 
values  are  compared,  and  a  new  improved  point  is  selected.  A 
finite  number  of  failures  to  locate  an  improved  point  results  in 
termination.  These  methods,  and  their  termination  criteria,  are 
for  the  most  part  heuristic,  unsupported  by  significant  mathematical 
statements  of  optimality.  Nevertheless,  they  can  usually  be 
expected  to  perform  well,  if  suboptimally ;  they  will  rather 
quickly  produce  a  "good"  function  value,  although  extreme  accuracy 
may  require  many  further  evaluations,  and  premature  stalls  are 
possible.  Exact  solutions  to  specialized  problems  (such  as  an 
extreme  point  solution  of  a  concave  minimization)  will  not  normally 
be  determined  precisely. 

These  last  observations  lead  to  an  important  caveat: 
if  a  program  with  special  structure  is  confronted  there  will  most 
likely  exist  a  more  effective  specialized  algorithm.  Of  course 
tracking  down  such  a  method,  especially  in  reliable  coded  fom,  may  not 
be  easy  (the  sources  in  [19,  20]  should  be  helpful  for  this 
purpose) .  Another  practical  point  is  this:  computer  time,  even 
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at  several  hundred  dollars/hour  is  still  cheaper  than  progreunmer/ 
analyst  time  at  $10-20/hour.  Even  if,  say,  a  routine  utilizing 
gradient  (and  possible  Hessian)  data  is  available,  these  deriva¬ 
tives  must  still  be  calculated  and  encoded.  The  savings  might 
typically  amount  to  a  few  tenths  of  a  second  of  execution  time, 
probably  not  a  cost  effective  trade-off.  An  important  possible 
exception  to  this  advice  occurs  when  the  purpose  of  the  optimizing 
code  is  to  serve  as  a  subroutine  of  some  large  program,  which  is 
to  process  data  in  real  time,  and  to  apply  some  control  law.  In 
such  a  case  speed  and  efficiency  are  likely  to  be  of  real  concern, 
and  specialized  software  may  be  required. 

A  review  of  several  popular  methods  of  pattern  search, 
including  the  complex  method  which  underlies  the  approach  described 
below,  is  given  in  Chapter  VII  of  [4]  and  in  [6,7].  We  also  take 
note  of  an  earlier  two-stage  heuristic  method  for  unconstrained 
optimization  announced  in  [23] .  It  consists  of  a  single  non-ran¬ 
dom  global  search  of  pre-determined  length  from  a  given  initial 
point  followed  by  a  local  pattern  search.  The  latter  is  of  the 
Hooke-Jeeves  type  [6,7],  with  a  modification  to  account  for 
curvature  in  the  gradient  path.  No  restart  or  other  interactive 
features  are  offered. 


III.  THE  PROPOSED  METHOD 

The  following  method  meets  the  criteria  we  have  just 
imposed:  it  is  simple,  versatile,  robust,  and  can  be  operated 

interactively.  It  requires  as  inputs  the  absolute  minimum  of 
information  that  is  needed  to  specify  a  mathematical  program  of 
the  type  (1) ,  (2) ,  plus  a  few  parameters  that  essentially  deter¬ 

mine  how  much  work  the  program  will  do.  We  also  emphasize  that 
the  proposed  njethod  requries  no  linear  algebraic  subroutines, 
such  as  matrix  inversions  or  linear  equation  solvers,  nor  does 
it  construct  derivative  approximations.  Our  code  can  thus  serve 
as  an  "off  the  shelf"  package  for  optimization,  model  fitting,  and 
nonlinear  equation  solving.  Several  examples  illustrating  the 
program's  use  for  such  applications  will  be  given  in  Section  V. 

In  essence  the  program  utilizes  the  fundamental  idea 
of  Box  [24],  wherein  a  random  finite  set  of  feasible  points  is 
generated  and  successively  deformed  (based  on  information  gleaned 
from  function  evaluations)  until  it  collapses  around  a  local 
minimum,  or  else  (on  occasion)  becomes  mired  in  an  area  where  the 
function  is  essentially  constant.  Several  modifications  of  this 
basic  iteration  have  been  made  to  prevent  various  possible  traps 
and  infinite  loops  that  are  possible  when  the  feasible  set  fails 
to  be  convex  or  even  connected.  Further  modifications  have  been 
made  to  provide  several  starting  and  continuation  options  for  the 
users,  as  well  as  printing  and  termination  options. 
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To  describe  the  program  in  greater  detail  it  is  convenient  to 
treat  separately  its  three  major  aspects: 

1)  the  basic  iteration; 

2)  termination; 

3)  initialization. 

We  will  then  make  some  comments  about  the  continuation  and  print¬ 
ing  options 

Referring  to  the  notation  of  (1) ,  and  assuming  that 
we  are  given  a  finite  subset  K  of  S,',  termed  a  "complex". 
Typically  the  size  of  K  (denoted  KPTS  in  the  program)  is  strictly 
between  n  and  2n;  unless  otherwise  instructed  our  program  sets 
KPTS  =  least  integer  ^  1.5n.  Three  points  of  K  are  now  singled 
out;  ::lo,  XNXTHI,  XHI,  so  that 

f (XLO)  =  min{f (X) ;XeK} 

£  ...  ^  f (XNXTHI)  < 

f(XHI)  =  max{f (X) ;XeK}. 

The  centroid  XC  of  {XeK;  X?^XHI }  is  now  computed  and,  for  the 
moment,  is  assumed  to  be  feasible  (i.e.,  XC  eS^).  Next  the  point 
XHI  is  reflected  through  XC  to  a  point  XNU  so  that  dist  (XNU,  XC) 
=  STEP* dist (XC,  XHI);  here  STEP  is  a  program  parameter  typically 
in  the  range  (1,2].  Thus 

XNU  =  XHI  +  (1+STEP)  (XC-XHI).  (3) 
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Now  if  it  happens  that  XNU  is  both  feasible  and  satisfies  f{XNU) 

<  f(XNXTHI)  we  define  a  new  complex  by  replacing  in  K  the  point 
XHI  by  XNU;  in  this  case  the  basic  iteration  is  complete. 

Should  either  of  the  two  requirements  on  XNU  fail,  XNU 
is  backed  halfway  towards  the  centroid  XC  along  the  line  through 
XC  and  XHI.  If  this  point  meets  both  requirements  we  substitute 
it  for  XHI  and  obtain  a  new  complex.  If  not,  we  halve  again. 

If  after  a  finite  number  (typically  5-10)  of  such  attempts  we 
are  still  unsucessful,  we  set  XNU  =  XC  and  commence  moving  XNU 
halfway  towards  XLO,  again  trying  to  meet  the  twin  requirements  of 
feasibility  and  lower  f-value.  If  a  finite  number  (typically  10- 
20)  of  such  attempts  are  still  unsuccessful,  the  algorithm  is 
terminated  if  the  last  move  resulted  in  feasibility  only.  If  not, 
we  reflect  the  latest  XNU  an  equal  distance  through  XLO  and  if 
both  requirements  still  cannot  be  met,  the  algorithm  is  again 
terminated.  In  all  these  cases  of  premature  termination  a  warning 
message  is  printed  indicating  either  an  infeasible  direction  (and 
suggesting  that  the  best  point  XLO  is  located  in  a  very  "thin" 
region  of  Q) ,  or  else  the  inability  to  decrease  the  function  value 
sufficiently,  in  which  case  the  algorithm  is  simply  "stuck". 

There  is  one  other  conceivable  difficulty  that  can 
arise  during  a  basic  iteration,  and  that  occurs  when  the  centroid 
XC  is  not  feasible.  Of  course,  this  could  only  happen  when  is 
not  convex.  In  this  case  we  again  set  XNU  =  XC  and  commence 
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moving  XNU  towards  XLO  as  just  described,  repeating  exactly  the 
above  steps.  Hence,  either  a  new  complex  is  obtained  or  else 
termination  ensues  with  the  appropriate  warning  message. 

Earlier  variants  of  this  basic  iteration  are  described 
in  the  original  source  [24]  and  in  [4,  20].  A  flow  chart  of  our 
version  appears  in  Fig.  1  (wherein  the  identification  Point — ►XNU 
should  be  made) . 

In  most  "reasonable"  optimization  problems  the  basic 
iteration  as  just  described  will  result  in  a  new  complex.  Con¬ 
sequently  we  can  expect  to  generate  a  sequence  of  complexes  and  so 
must  decide  when  to  cease.  Roughly  we  should  do  so  when  no  further 
progress  is  being  achieved.  In  practice  we  evaluate  this  con¬ 
dition  by  means  of  the  absolute  and  relative  oscillation  of  f  on 
the  complexes.  If  either  of  these  quantities  drops  under  given 
thresholds  for  a  finite  number  (typically  5)  of  complexes  in 
succession,  the  algorithm  terminates.  Otherwise  a  new  basic 
iteration  is  carried  out.  Of  course  an  upper  bound  on  the  number 
of  iterations  must  also  be  supplied. 

Thus,  assuming  that  new  complexes  can  be  generated  we 
control  termination  with  four  program  parameters:  ABSTOL,  RELTOL, 
NTOL,  and  NLOOPS.  If  for  a  sequence  of  length  NTOL  of  complexes 


i-g 


The  basic  iteration 


^i  +  1'  ^i+2'“*'  ^i+NTOL 


f(Xni)  -  f(XLO)  <  ABSTOL, 


or  failing  this,  if 


f(XHI)  -  f(XLO)  £  RELTOL*  1  f  (XHI)  I , 


(4) 


(5) 


the  program  terminates  with  a  cons-’ergence  assertion.  Otherwise, 
it  terminates  with  a  warning  when  the  number  of  basic  iterations 
exceeds  NLOOPS.  These  termination  procedures  are  displayed  in  Fig. 

To  initialize  the  optimization  process  we  must  construct 
a  first  complex.  Our  program  permits  this  to  be  done  in  three 
ways ; 

a)  by  direct  assignment; 

b)  randomly  about  a  given  feasible  point; 

c)  choosing  the  best  KPTS  points  from  a  preliminary 
random  search  of 

Option  a)  might  be  used  when  the  geometric  structure  of  was 
known  and  simple  (e.g.,  Q=  {  (x^ ,  . .  . , x^)  :  0£x^,2x^£l),  and  no 
information  about  the  minimum  of  f  over  Q.  was  available.  Option 
b)  is  valuable  when  there  is  a  priori  information  about  f  to  the 
extent  that  we  believe  that  for  some  f(x^*^^)  is  close  to 

min  f(H)  .  The  remaining  KPTS-1  points  for  the  initial  complex 
are  then  produced  sequentially  by  first  drawing  a  point  randomly 
within  the  region  defined  by  the  explicit  constraints  {x; 

£b^,  l£i£n}  in  (2),  and  then  retracting  it  by  halves  towards  x^®^ 
until  it  becomes  feasible.  If  a  finite  number  (typically  10~20) 
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Pig.  2.  Algorithm  termination 


of  these  retractions  fails  to  result  in  feasibility,  we  begin 
again  with  a  new  random  point. 

In  the  remaining  circumstances  we  are  effectively  ignorant  of  both 
the  geometry  of  and  the  behavior  of  f.  If  so,  we  utilize  option 
c)  by  making  an  initial  random  search  of  and  retaining  the  best 
KPTS  points  to  form  the  initial  complex.  The  only  complication 
here  is  the  number  of  random  points  (denoted  NRANPT  in  the  program) 
to  utilize.  Clearly  we  must  have  NRANPT  ^  KPTS,  but  also  in 
practice  we  must  confront  the  trade-off  between  the  cost  in  terms 
of  function  evaluations  of  taking  NRANPT  large  vs.  the  cost  in 
terms  of  low  probability  of  being  close  to  the  minimum  if  NRANPT 
is  small. 


One  way  to  sensibly  determine  a  value  of  NRANPT  is  to 
require  a  certain  probability  of  placing  at  least  k  points  inside 
a  small  neighborhood  V  of  the  global  minimum  of  f  in  fi.  If  the 
relative  volume  of  V  in  n  is  a,  then 

Prob  k  points  in  V  in  N  tries) 


=  1 


=  1 


>  1 


N-i 


S  (*)‘* 

,,  ,N  .,k-l  /  a  \ 

(1-a)  N  expf^^^J 


N1 

(N-i) ! 


so  that  to  make  this  probability  exceed  0  it  suffices  to  choose 
N  so  large  that 


18 


.N  a  ^  I  n 

(1-a)  N  exp  £  1-3. 

In  the  special  case  where  k=l,  the  "exp"  term  can  be  dropped 
from  the  left  hand  side. 

To  illustrate,  let  us  take  V  be  to  a  cube  of  side  length 
2r  centered  at  the  global  minimum.  In  general  V  need  not  exist 
if  the  minimum  is  too  near  the  boundary  of  Q.  However,  if  there 
are  no  implicit  constraints,  or  if  we  can  be  sure  for  other 
reasons  that  the  minimum  is  well  inside  ^2,  then  it  makes  sense 
to  discuss  V.  Now  a=  (2r) ’^/vol  (ft )  and  if  k=l  we  need  to  choose 
N  so  that 


N  >  >  i2a(^  .  -  vokr). 

*—  log  (1— ot)  —  —  ot  (2r)^ 

The  least  such  N  are  displayed  in  Fig.  3  for  nominal  values  of  3 
and  n,  with  r  set  =  0.1  and  vol{fi)=l.  One  can  easily  note  a  very 
prominent  "curse  of  dimensionality"  here.  This  concludes  our 
discussion  of  the  initialization  step  in  the  program. 

The  progress  of  the  algorithm  when  applied  to  a  par¬ 
ticular  problem  can  be  monitored  as  desired  through  the  use  of 
four  print  options.  These  are  set  by  assigning  to  a  variable 
IPRINT  values  =  0,1, 2, 3.  The  0  value  supresses  all  printing; 
this  might  be  appropriate  vdien  the  optimization  is  serving  as 
a  subroutine  to  a  larger  program.  The  1  value  causes  only  the 
final  answer  (best  point  and  least  function  value)  and  machine 
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1 


TR-576(3) 


2 

5 

10 

in 

• 

o 

18 

2166 

6,769,016 

0.9 

58 

7196 

22,486,183 

0.95 

75 

9362 

29,255,198 

Fig.  3.  Least  niimber  of  random  samples  to 
give  probability  B  of  at  least  one  sample 
falling  in  cube  of  radius  0.1  within  a 
unit  n-volume. 

execution  time  to  be  displayed.  The  2  value  results  in  display  of 
the  best  initial  complex  point  and  corresponding  function  value 
followed  by  each  new  improved  point,  the  corresponding  lower  func¬ 
tion  value,  and  the  loop  number  w  en  this  improvement  occurred. 
Finally,  the  3  value  causes  the  entire  initial  complex  to  be  dis¬ 
played,  followed  by  further  information  for  every  iteration, 
whether  or  not  an  improvement  was  achieved.  Specifically,  the  new 
point  and  its  function  value  are  displayed  along  with  the  index  of 
the  point  in  the  previous  complex  which  it  replaces,  and  the 
number  of  function  evaluations  required  to  obtain  the  new  point. 

It  remains  to  discuss  the  continuation  options.  Recall 
that  the  program  will  terminate  for  one  of  three  reasons:  con¬ 
vergence  criterion  met,  number  of  iterations  loops  exceeded,  or 
prematurely,  because  of  failure  to  construct  a  new  complex.  Use 
of  the  IPRINT  =2  or  3  option  will  have  enabled  the  user  to  gain 
an  impression  of  the  progress  made  and  its  rate  since  initializa¬ 
tion.  If  this  progress  is  satisfactory  but  the  oonvergetx:e  criterion 
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is  unmet,  the  user  will  probably  wish  to  continue  from  the  most 
recent  complex.  On  the  other  hand,  if  the  algorithm  appears  to 
be  struggling,  the  user  is  also  given  the  option  of  restarting 
around  the  best  point  discovered.  That  is,  the  best  point  is  now 
denoted  and  initialization  option  b)  (defined  on  p.  16)  is 

implemented. 
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IV 


PARAMETER  AND  SUBROUTINE  SPECIFICATIONS 


The  (FORTRAN)  program  consists  of  a  main  progreun,  seven 
general  subroutines,  and  two  user  supplied  subroutines.  In  greater 
detail  these  are  described  next. 

SUBROUTINE  BOX  is  called  by  the  main  program  to  set  up 
the  initial  complex,  call  the  basic  iterative  subroutine  GOBOX,  and 
print  the  final  answer. 

SUBROUTINE  GOBOX  is  the  basic  iterative  subroutine;  it 
accepts  an  initial  complex  and  iteratively  constructs  new  ones 
until  termination.  As  necessary  it  calls  the  special  purpose  sub¬ 
routines  MOVE,  FEASBL,  CENTER,  and  the  function  evaluator  FUNC. 

SUBROUTINE  MOVE  repeatedly  defines  a  new  point  halfway 
between  two  given  points  until  certain  termination  criteria  are 
met. 

SUBROUTINE  FEASBL  checks  that  a  given  vector  belongs  to 
sets  0.  of  the  form  (2)  .  If  not  it  attempts  to  reposition  this 
vector  so  as  to  meet  all  constraints. 

SUBROUTINE  CENTER  computes  the  centroid  of  a  complex 
less  one  point. 

SUBROUTINE  GBO  defines  an  all  random  initial  complex 
according  to  initialization  option  c) . 

SUBROUTINE  GDI  defines  a  random  complex  to  include  one 
given  feasible  point  according  to  initialization  option  b) . 

SUBROUTINE  FUNC  computes  the  value  of  a  given  function 


at  a  given  point  in  its  domain. 

SUBROUTINE  IMPLCT  defines  the  various  implicit  constraint 
functions  appearing  in  (2). 

Progreim  parameters  and  control  variables  are  user  supplied 
as  needed  through  a  NAMELIST  specification  statement.  It  assigns 
the  neune  "GO"  to  refer  to  the  following  list  of  variables  and 
array  names ... 

Problem  description: 

NX:  number  of  independent  variables 

NIC:  number  of  implicit  constraint  functions  g^  in  (2) 

MAXMIM:  set  *  +1(-1)  to  maximize  (minimize) 

XMIN:  array  of  lower  bounds  for  independent  variables 

XMAX:  array  of  upper  bounds  for  independent  variables 

XIMIN:  array  of  lower  bounds  on  constraint  functions 

XIMAX:  array  of  upper  bounds  on  constraint  functions 

Basic  iteration  controls; 

KPTSET:  used  to  prescribe  number  of  points  in  complex 

NICUTS:  upper  bound  on  number  of  halfway  moves  towards  centroid 

N2CUTS;  upper  bound  on  number  of  halfway  moves  away  from  centroid 

STEP:  step  size  of  reflection  in  basic  iteration  (see  (3)) 

NLOOPS;  upper  bound  on  number  of  basic  iterations 

convergence  parameter  =  number  of  consecutive  iterations 
without  sufficient  progress. 


NTOL: 


ABSTOL; 


convergence  parameter  (see  (4)  ) 
RELTOL:  convergence  parameter  (see  (5)) 


Initialization  and  run  parameters: 


I  RUN: 
IPRINT 
MAX  RAN 
NRANPT 

SEED: 

XO: 

XXO: 


program  operation  control;  usage  defined  in  section  V 

output  control;  usage  defined  in  section  III 

bound  on  number  of  attempts  to  establish  initial  complex 

total  number  of  random  draws  from  feasible  set  to  define 
initial  all-random  complex 

seed  used  in  system  uniform  random  number  generator 
initial  feasible  point 
initial  feasible  complex 


Of  these  variables  it  is  imperative  that  the  user 
assign  values  to  the  first  five  and,  if  NIC  >  0,  to  the  next  two 
also.  Having  done  so  the  program  will  run  using  the  default  set¬ 
tings  of  the  remaining  parameters;  in  particular  an  all-random 
initial  complex  will  be  chosen. 

In  practice,  the  user  will  want  to  consider  the  appro¬ 
priate  values  for  the  initialization  and  convergence  parcimeters. 
According  to  the  initialization  option  chosen,  one  of  XO,  XXO  or 
NRANPT  must  be  defined.  The  user  will  also  probably  need  to  assign 
relevant  values  to  the  convergence  tolerance  values  ABSTOL  and 
RELTOL,  and  the  iteration  bound  NLOOPS.  By  default,  ABSTOL=0. ,RELT0L= 
0.000001  and  NLOOPS=500.  Also,  by  default,  STEP=1.5, 
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KPTS=IFIX  (1.5*NX+0.5) ,  N1CUTS=8,  N2CUTS=16,  and  IPRINT=1.  The 
use  of  the  program  control  IRON  is  described  in  the  next  section, 
along  with  several  examples. 

As  already  noted  the  STEP  parameter  should  satisfy 
1.0  <  STEP  £  2.0.  Ideally  one  would  like  STEP  larger  initially  to 
make  rapid  progress  towards  the  solution  point  and  smaller  as  this 
point  is  approached.  In  general,  larger  values  of  STEP  can  result 
in  infeasible  reflected  points  and  consequent  loss  of  time  in 
retreating  towards  the  centroid,  while  small  values  of  STEP  result 
in  slow  progress  and  increased  iterations.  However,  as  the  length 
of  a  reflection  step  is  roughly  proportional  to  the  diameter  of  the 
associated  complex,  the  step  lengths  automatically  decrease  as  the 
algorithm  progresses  and  the  complexes  shrink.  Hence  algorithm 
performance  is  relatively  indifferent  to  moderate  changes  in  STEP 
and  STEP  =  1.5  has  proved  to  be  a  satisfactory  default  value. 

Finally,  there  is  the  matter  of  "relevant  values"  for  the 
tolerances  ABSTOL  and  RELTOL.  In  general,  only  one  of  these  needs 
be  positive.  If  the  user  has  a  good  idea  of  the  order  of  magnitude 
of  the  optimal  function  value  (as,  for  instance,  in  Example  4  of 
the  next  section) ,  then  ABSTOL  should  be  set  according  to  the  final 
accuracy  desired.  If,  on  the  other  hand,  the  user  has  no  reliable 
guide  to  the  magnitude  of  the  final  function  values,  we  suggest  the 
default  values  of  these  two  toleremce  peuranteters.  Ihese  cases  aite  illustrated 
in  Exanples  5,  6  belcw  v4iere,  in  fact,  even  annaller  values  of  RELTOL  are  anployed. 


V.  PROGRAM  OPERATION  AND  EXAMPLES 


To  utilize  the  progreun  for  the  solution  of  a  particular 


optimization  problem  of  the  form  (1) ,  (2) ,  the  user  should  do  the 


following; 

1) 


2) 

3) 


IRON  =  0 

1 

2 

3 

4 


5: 


6 : 
7; 


define  the  objective  function  f  in  SUBROUTINE  FUNC, 
and  the  constraint  functions  gj  in  SUBROUTINE  IMPLCT. 
In  the  latter  the  notation  XI  (J)  is  used  for  g ^ , 
with  J=l,  2,  NIC; 

alter  the  DIMENSION  statement  in  the  main  program, 
if  necessary,  to  accomodate  values  NX>10  or  NIC>9. 
Compile  and  run,  then: 

input  desired  values  of  the  relevant  NAMELIST 
variables,  finally  assigning  a  value  to  the  control 
IRUN  from  among  the  following  options. 

stop 

begin  with  user  specified  complex  XXO 

begin  with  user  specified  feasible  point  XO 

begin  with  all'-random  complex  (specify  NRANPT) 

continue  with  further  iterations  from  most  recent 
complex 

restart  as  in  IRUN  =  2  with  XO  set  equal  current 
best  point 

reset  all  NAMELIST  variables  to  default  values 
list  current  value  of  all  NAMELIST  variables 


We  now  illustrate  the  operation  of  the  program  and  the 
results  obtained  on  a  variety  of  examples.  The  actual  outputs 
reported  below  result  from  operation  of  a  double  precision  FORTRAN 
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version  of  our  method  in  CMS  on  an  Amdahl  470/V7  mainfraune  computer. 
A  listing  of  this  version  appears  in  the  Appendix. 

2  2  2 

Excunple  1.  Minimize  100  (X2-Xj^  )  +  (1-Xj^)  . 

This  is  a  famous  test  problem  associated  with  the  neune  of  Rosen- 
brock.  It  is  an  unconstrained  minimization;  the  objective  function 
surface  is  notable  for  a  narrow  deep  curving  valley.  The  true 
solution  is  clearly  0  at  (1,1).  We  implement  the  progreim  by 
setting  NX=2,  NIC=0,  XMIN=2*-2.,  XMAX=2*2.  (thereby  constraining 

(Xj^2)  cube  defined  by  |x^|£2),  and  MAXMIN  =  -1.  We 

report  in  Fig.  4  below  the  results  of  three  sets  of  10  runs  with 
the  indicated  values  of  ABSTOL,  RELTOL,  and  STEP.  (In  all  the 
examples  the  default  value  STEP  =  1.5  is  used  unless  otherwise 
noted.)  In  all  cases  the  conventional  starting  value  XO  =  (-1.2, 
1.0)  was  used  (.IRON  =  2)  .  For  this  exar?>le,  and  in  general  v*ien  we  can 
reasonably  guess  at  the  magnitude  of  the  optinal  function  value,  we  should  set 
RELTQL  =  0.0  and  chcx>se  ABSTOL  according  to  the  desired  accurcx:y.  We  did  so 
in  the  second  set  of  runs  here,  desiring  that  F  be  reduced  belcw  10  The 
third  set  is  the  same  except  for  STEP  =  1.3  (the  original  Box  reocranendation) ; 
in  this  case  we  note  a  slightly  poorer  and  more  erratic  result. 


Fuixtion  Pinal 

Evaluations  Function  Value 


ABSTOL  =  0.0 

RELTOL  =  0.000001 

916 

(323) 

ABSTOL  =  5.0D-11 

360 

RELTQL  =  0.0 

(53) 

kES2!SESiiHB 

ABSTOL  =  5.0D-11 

317 

6.773  D-09 

RELTOL  =  0.0 

STOP  =  1.3 

(63) 

(18.333)0-09 

Fig.  4.  Means  and  (standard  errors)  for  the  Rosenbrock  function  minimization. 
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V, 


We  note  that  since  feasible  points  are  readily  available  in 
this  example,  the  pure  random  start,  being  wasteful  of  function 
evaluations,  is  avoided. 

2  2 

Example  2.  Maximixe  Xj^  +4Xj^X2+7x2  over  the  set  f2={  (Xj^,X2)  : 
0^x^£l,  -l£Xj^+2x2£l  /  -l_f^3Xj^-4x2£l } .  This  is  a  concave  quadratic 
program  over  the  pentagonal  region  with  exact  solution  1.48  at 
the  vertex  (0.2,  0.4).  We  set  NX=NIC=2,  XMIN=2*0.1,  XMAX=2*1.1, 
XIMIN=2*-1.1,  XIMAX=2^1.1,  ABSTOL=RELTOL=. 000001 ,  and  made  ten 
runs  starting  from  XO=(0.3,  0.2).  The  results  were 


function 

function 

solution 

evaluations 

value 

point 

mean 

54 

1.478883 

(.200004,  .399998) 

(standard  error) 

(10) 

(0.000012) 

((.000010,  .000005)). 

2  4  6  2  4 

Example  3.  Minimize  4Xj^  -2.1Xj^  +Xj^  /3+Xj^X2-4x2  +4x2  over 
the  rectangle  =  {  (x^^,  X2)  :  |xj^|;<2.5,  |x2l£l.5}.  This  is  another 

famous  test  problem,  the  objective  function  being  Jcnown  as  the  "six 
hump  camel  baclc  function"  [17]  .  It  is  a  challenge  for  local 
gradient-utilizing  methods  because  of  the  presence  of  an  assortment 
of  stationary  points;  6  local  minima,  2  local  maxima,  and  7  saddle 
points.  Published  numerical  tests  of  other  global  optimization 
procedures  [17]  give  a  global  minimum  value  of  -1.0316  at  the 
points  (0.08984,  -0.71266)  and  (0.71266,  -0.08984),  at  a  cost  of 
72-800  function  evaluations.  We  set  ABSTOL=RELTOL=0 . 0001  and  made 
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5  runs  each  with  KPTS=3,  KPTS=4,  starting  in  all  cases  at  X0=(0,0) 
(IRUN=2) .  The  results  were 


function  evaluations  function  value 


KPTS=3 

132 

-1.031626 

(21) 

(0.000001) 

KPTS=4 

130 

-1.031617 

(14) 

(0.000012) 

We  note  eibout  the  seime  amount  of  work  involved  with  the  larger 
complex,  but  also  a  slightly  greater  variability  in  the  final 
answer . 

Example  4 .  Solve  the  nonlinear  system  of  equations 

-  3  3 

2Xj^  X2  =  X2 
2 

6Xj^  ”  *2  +  *2  ~ 

By  sketching  the  corresponding  pair  of  curves  in  the  (Xj^,X2)  - 

plane,  one  can  easily  verify  a  pair  of  exact  solutions  at  (0,0)  and 

(2,4)  and  one  other  solution  near  (1.5,  -  3.0).  To  determine  this 

third  solution  exactly  we  try  to  minimize  the  sum  of  the  squared 

332  2  ^ 

residuals;  that  is,  minimize  (2Xj^  *2”*2  ^  ‘*‘*2^  ’ 

least  value  is  of  course  0,  and  the  solution  point  will  be  our 
unknown  third  solution.  We  set  ^“{(Xj^,  X2)!  -  4£X2£-2}, 
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X0=(1.3,  -  3.0),  and  ABSTOL  =  l.OD-13,  RELTOL  =  0.0,  (attempting  to 
obtain  six  or  seven  significant  figures).  The  results  of  five 


runs  were 


function  evaluations 

solution  point 

mean 

291 

(1.464352,  -2.506013) 

(standard  error) 

(24) 

(O.OOOOOO,  0.000000) 

In  all  cases  the  objective  function  was  reduced  to  less  than 
5.0D-12,  with  an  average  final  value  =  8.85D-13. 

2  2  2 

Exaunple  5.  Minimize  x^^  +X2  +X2  over  the  region 
fi={(Xj^,  x^/  x^)  :  jx^l^lO,  3_<Xj^X2X2,  3£x^+X2-X2}.  This  problem  is 
unusual  because  is  disconnected:  it  consists  of  three  components 
each  of  equal  distance  from  the  origin.  Hence  there  are  three 
distinct  solution  points,  each  of  which  is  a  boundary  point.  These 
points  can  be  obtained  via  the  Kuhn-Tucker  conditions  and  analysis 
of  the  possible  sign  patterns  of  the  coordinates,  viz.,  (+,+,+), 
(+,-,-),  and  (-,+,-).  Thus  the  first  point  must  obey  the  conditions 


X 


*1  = 
*1*2*3 


1+*2"*3 


X 


2 


3 

3. 


2 

This  leads  to  the  cubic  equations  x  {2x-3)=3  for  Xj^;  its  solution 
=  1.910820  =  *2“®*  Therefore,  x^=2x^-3  =  0. 8216404b.  By  symmetry, 
the  other  solution  points  are  (a,  -b,  -a)  and  (-b,  a,  -a) ,  and  the 


common  minimum  function  value  =  7.977559 


This  example  is  meant  to  typify  the  situation  wherein  we 
are  ignorant  of  the  geometry  of  the  feasible  set  In  such  cases 
we  must  resort  to  the  pure  random  start  option  (IRUN=3) .  Setting 
NRANPT  =  500,  ABSTOL  =  0.,  and  RELTOL  =  l.OD-08,  the  program  easily 
reveals  the  three  distinct  solution  points.  The  results  of  ten 
runs  were 


function  evaluations 

function  value 

mean 

1043 

7.977583 

(standard  error) 

(253) 

(0.000046) 

Eight  of  the  ten  runs  actually  resulted  in  function  values  ^7.977560. 

2  2  2  2  2 

Example  6.  Minimize  100(x2“Xj^  )  +  (1-Xj^)  +  90(x^-X2  ) 

+  (1-X3)2  +  10.1[(X2-1)^  +  (x^-l)^]  +  19.8(X2-1) (X4-I) ,  over  the 
four-dimensional  cube  n={  (Xj^,X2,X3,x^) :  |x^  |£10}.  This  is  another 
test  problem  associated  with  the  ncune  of  Wood  [6,  p.  403];  it  is 
designed  to  have  a  non-optimal  stationary  point  with  a  corresponding 
function  value  of  approximately  8.0  that  can  cause  premature 
convergence.  The  optimal  solution  is  clearly  0  at  (1,1, 1,1).  A 
starting  value  XO= (-3,-1, -3,-1)  is  suggested.  Our  program  handled 
this  problem  easily  with  no  premature  stalls.  The  results  of  ten 
runs  with  ABSTOL=0.,  RELTOL=1.0D-10  (the  value  of  the  objective 
function  was  increased  by  1)  uniformly  yielded  function  values 


<  1.0000000005.  with  an  average  of  1145  (198)  function  values 


Example  7 .  Our  final  example  is  a  nonlinear  regression 

taken  from  [24] .  The  problem  is  to  model  the  resistance  (R)  of 
a  thermistor  as  a  function  of  temperature  (T)  via  the  model 


R  =  a  exp 


(tvt)- 


Specifically  we  want  to  assign  values  to  the  parameters  a,b,c 
so  as  to  achieve  the  best  fit  to  the  data 


T 

R 

50 

34,780 

55 

28,610 

60 

23,650 

65 

19,630 

70 

16,370 

75 

13,720 

80 

11,540 

85 

9,744 

90 

8,261 

95 

7,030 

100 

6,005 

105 

5,147 

110 

4,427 

115 

3,820 

120 

3,307 

125 

2,872 
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We  do  so  by  minimizing  the  mean  squared  residual 


The  source  cited  reports  that  this  problem  caused  considerable 
difficulity  to  the  algorithms  being  tested.  In  general  we  note 
that  the  special  structure  of' nonlinear  least  squares  problems  can 
be  utilized  in  the  design  of  specialized  algorithms  such  as  the 
methods  of  Gauss-Newton  or  Levenberg-Marquardt  [25].  For  the 
latest  in  such  algorithms,  see  [26] .  These  may  certainly  be 
expected  to  be  far  more  efficient  than  our  present  all-purpose 
method.  Nevertheless,  it  is  interesting  to  test  our  method  on 
such  a  problem,  recalling  particularly  the  discussion  of  the 
computer  time  vs.  analyst  time  trade-off. 

The  suggested  initial  values  are  XO=(0.2,  4000,  250.), 
the  corresponding  function  value  being  41,153.  We  set  STEP=2. 
and  ABSTOL  =  .0001.  =  RELTOL  .  We  also  introduce  the  artifical 
constraint  gj^(a,b,c)  =  b/(50+c)  £  70  in  order  to  avoid  overflow 
problems  in  the  exponential.  The  outcomes  of  ten  runs  were 
classified  as  success  or  failure  according  as  convergence  was 
or  was  not  achieved  with  at  most  five  restarts  (IRUN=5) .  The 
results  were 
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number 

function 

evaluations 

function 

value 

CPU  time 
(sec. ) 

6480 

9.3788 

mm 

success  6 

(1250) 

( .0010) 

3031 

267.74 

1.24 

failure  4 

(1340) 

(90) 

(.70) 

The  mean  solution  point  for  the  six  successes  was 

(.005612,  6181.00,  345.210) 

with  standard  errors  (.000006,  0.85,  0.030);  this  may  be  compared 
with  the  optimal  solution  reported  in  [24): 


(.005609,  6181.,  345.2) 


VI 


CONCLUSIONS 


i 

\ 

X 

% 

♦ 


The  numerical  practice  of  nonlinear  multivariate 
optimization  is  an  important,  difficult,  and  much  studied  subject. 
Numerous  algorithms  have  been  proposed,  some  exotic,  some  effective, 
most  requiring  special  hypotheses  on  the  objective  and  constraint 
functions.  We  have  presented  here  an  extremely  simple  and  robust 
procedure  for  handling  the  most  general  nonlinear  inequality  con¬ 
strained  optimization,  and  demonstrated  its  operation  and  effective¬ 
ness  on  a  variety  of  test  problems  of  low  dimension.  The  method 
utilizes  the  basic  Box  complex  iteration  with  modifications  to 
avoid  traps.  The  computer  implementation  provides  a  variety  of 
initialization  and  continuation  features  and  is  designed  for 
interactive  use.  While  problems  of  a  special  nature  can  be  solved 
more  efficiently  with  specially  formulated  algorithms,  in  a  certain 
practical  sense  our  method  is  reasonably  competitive  with  these, 
and  moreover  constitutes  a  safe  reliable  procedure  for  use  on 
problems  with  little  or  no  exploitable  structure.  Examples  of 
the  latter,  on  which  our  method  has  been  successfully  applied, 
include  minimizing  a  discontinuous  cost  function  subject  to  a 
communications  network  reliability  constraint,  and  minimizing  a 
two-stage  survival  probcibility  function  to  achieve  an  optimal 
attack  against  a  given  layered  ballistic  missile  defense  system. 


'.a.  w. 
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This  appendix  contains  the  complete  FORTRAN  listing  of  the 
optimization  program  described  above.  The  data  for  the  function 
and  implicit  constraint  portions,  SUBROUTINES  FUNC  and  IMPLCT 
respectively,  correspond  to  Example  7  in  Section  V. 


IMPLICIT  REAL*8  <A-H»  0-2) 

DIMENSION  XXdOf  15)  fXSTARdO)  »XMIN(10>fXMAX(10)  »X0<10) 

DIMENSION  XIMIN<9)fXIMAX<9)»XI(9)  »FFd5)  »XXOdO»  15)  »XdO) 

COMMON  /BOXCOM/  STEP»RELTOL» ABSTOLrNTOL»NlCUTS»N2CUTSfNLOOPS 
«  f I PR I NT  f  MAXMIN » SEED  f  NRANPT » MAXRAN 

REAL»8  LXMIN»LXMAX»LXIMIN»LXIMAX»LXO»LXXOrLBLANK 
NAMELIST  /GO/  NXf NIC»MAXMIN»XMIN»XMAX»XIMIN»XIMAX» 

*  KPTSET  f  NICUTS  »  N2CUTS  »  STEP  r  NLOOPS  f  NTOL » ABSTOL » RELTOL » 

«  I RUN  f I PRINT  f  NRANPT  f  MAXRAN  f  SEED  f  XO  f XXO 

DATA  LXMIN.LXMAX  /8H  XMIN  »8H  XMAX  / 

DATA  LXIMIN»LXIMAX  /8H  XIMIN  f8H  XIMAX  / 

DATA  LXO»LXXOfLBLANK  /8H  XO  »8H  XXO  f8H  / 

C 

C  PROGRAM  NEUBOX  USES  SUBROUTINE  BOX  TO  SOLVE  THE  PROBLEM 
C  MAXIMIZE  MAXMIN*FUNC<X) 

C  SUBJECT  TO  XMINd)  .LE.  Xd)  .LE,  XMAX(I)  fI  =  lf...rNX 

C  XIMIN(J)  .LE.  XKJ)  .LE.  XIMAX(J)  fJ=lf...»NIC 

C  WHERE  XKJ)  IS  THE  VALUE  OF  THE  JTH  IMPLICIT  CONSTRAINT  FUNCTION 
C  EVALUATED  AT  THE  POINT  X»  AS  COMPUTED  IN  SUBROUTINE  IMPLCT. 

C 

C  THE  USER  MUST  SUPPLY  SUBROUTINE  FUNC(X»NX»F)  WHICH  CALCULATES  Fr 
C  THE  FUNCTIONAL  VALUE  OF  A  POINT  X»  AND  A  SUBROUTINE 
C  IMPLCT<XfNX»XI»NIC)  which  calculates  XI »  THE  NIC  IMPLICIT 
C  CONSTRAINT  VALUES  CORRESPONDING  TO  THE  POINT  X. 

C 

C 

C  SET  PARAMETERS  TO  THEIR  DEFAULT  VALUES. 

C 

100  KPTSET=0 
N1CUTS=8 
N2CUTS-16 
STEP=1.5 
NL00PS=500 
NT0L=5 
ABSTOLaO. 

RELTOL=. 000001 

IRUN=3 

IPRINT®! 

NRANPT®0 

MAXRAN®100 

SEED-184287807 

C 
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C  READ  NAMELIST  »G0  DATA  DEFINING  PARAMETERS  FOR  NEXT  RUN 

C 

200  WRITE<6f9) 

9  FORMAT  ( 'lENTER  NAMELIST  8,G0  INPUT  (IRUN=7  LISTS  ALL)') 
READ(5»G0) 

IFdRUN.LE.O)  GO  TO  1000 
IF(IRUN-6)  300* 100 » 400 
C 

C  CALL  BOX  TO  OPTIMIZE  FOR  THE  CURRENT  PARAMETERS 
C 

300  KPTS~NX+(NX+1 )/2 

I F ( KPTSET . GE . 2 )  KPTS=KPTSET 

CALL  BOX(XX*NX*KPTS»NIC*XMIN*XMAX*XIMIN*XIMAX 
*  *FF.XI *XSTAR»FSTAR»X*XO*XXO*IRUN) 

GO  TO  200 
C 

C  PRINT  THE  CURRENT  VALUES  OF  THE  PARAMETERS 


C 

400  URITE(6*1)  NX*NIC*MAXMIN 

1  FORMAT ( 'OCURRENT  VALUES  OF  PARAMETERS  ARE}'*/* 


*  '0' *10X* 'PROBLEM  DESCRIPTION'*/* 

*  '  NX*NIC*MAXMIN  =  '*315) 
WRITE<6*2)  LXMIN*  <XMIN(I) *I*lfNX) 


WRITE<6*2)  LXMAX*  (XMAX(I)*I==1*NX) 

WRITE (6* 2)  LXIMIN*  <XIMIN<I)*I  =  1*NIC) 

WRITE (A *2)  LXIMAX* (XIMAX( I ) *  1  =  1 *NIC ) 

2  F0RMAT(A8*'=  ' * 1P10E12 . 4*/ * < lOX * 1P10E12 . 4) ) 

WR I TE  <  6  *  3 )  KPTSET  *  N1 CUTS  *  N2CUTS  *  STEP  *  NLOOPS  *  NTOL  *  ABSTOL  *  RELTOL 

3  FORMAT< '0' *10X* 'BOX  PARAMETERS'*/* 

*  '  KPTSET*N1CUTS*N2CUTS*STEP  =  ' *3I5*F12.6*/* 

*  '  NLOOPS* NTOL* ABSTOL* RELTOL  =  ' * 215 * 1P2E12 , 3 ) 

WR I TE ( 6  *  4 )  I RUN  *  I PR I NT  *  NRANPT  *  MAXR AN  r  SEED 

4  FORMAT< '0' *10X* 'RUN  INITIALIZATION  PARAMETERS '*/ » 

»  'IRUN*IPRINT*NRANPT*MAXRAN*SEED  =  ' *2I4»2I7»3X»Z16) 

WRITE<6*2)  LX0*(X0<I)*I=1*NX) 

WRITE (A* 2)  LXXO* (XX0<I*1>*I=1»NX) 

DO  440  J=2*KPTS 

440  WRITE(A*2)  LBLANK* <XXO( I  * J) *  1  =  1 *NX) 

GO  TO  200 
C 

1000  STOP 


END 
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SUBROUTINE  BOXCXXt NXtKPTSt NTCfXHINf XHAXtXIHINt XIMAX»FF»XT 
«  fXSTARf FSTAR»X»XOfXXO» IRUN) 

IMPLICIT  REAL*8  (A-Hf  0-Z) 

DIMENSION  XX(NX»KPTS) »XMIN(NX) »XHAX(NX) tXIHIN(Nin) »XIMAX(NIC) 
DIMENSION  FF<KPTS)  fXKNIC)  fXSTARINX) 

DIMENSION  X(NX) .X0(NX)»XX0(NXfKPT8) 

COMMON  /BOXCOM/  STEP » RELTOL  » ABSTOl. » NTOL  » N1  CUTS » N2CUTS  ?  NLOOPS 
$  »IPRINT»HAXHIN»SEED»NRANPT 

REAL»R  DOATE»DTTME 

BOX  SETS  UP  THE  INITIAL  COMPLEX  THEN  CALLS  OOBOX  TO  FIND  THE 
OPTIMAL  NX-VECTOR  XSTAR  AND  ITS  FUNCTIONAL  VALUE  FSTAR. 

60  TO  ( lOOf 200f 300»400»500) > IRUN 

IRUN=1  ....  SET  INITIAL  XX  TO  USER-DEFINED  XXO 

100  URITE(Afl)  KPTSfSTEP 

1  FORMAT< 'OENTERING  BOX  WITH  DEFAULT  XX' »/ 

♦  f'  KPTSfSTEP  »  'fISfFlO.6) 

DO  150  J=1.KPTS 
DO  140  1=1. NX 
140  XX(I..J)=XXO(I.  J) 

CALI.  FUNC<XX<l.J>fNX.FF(J)) 

150  CONTINUE 
NFUNr=KPTS 
GO  TO  900 


IRUN=2  ....  SET  INITIAL  XX  RANDOMLY  AROUND  (FEASIBLE)  XO 

200  WRITE(6.?)  SEED. KPTSfSTEP. XO 

2  FORMAT! 'CENTERING  BOX  WITH  RANDOM  XX.  SEED  »  '.D13.5./ 

*  .'  KPTSfSTEP  ■  '.I5.F10.6./ 

*  .'  XO  ='»1P9E12.4./.(9X»1P9E12.4)) 

CALL  OBI (XX.NX.KPTS.XO.FF.XMJN.XMAX.XIMIN.XIMAXrXI 
«  .NIC.IOK) 

NFUNC»KPTS 

IFdOK.GT.O)  00  TO  900 
GO  TO  1000 

IRUN=S  ....  SET  INITIAL  XX  RANDOMLY  IN  FEASIBLE  REGION 

300  WRITE<A.3)  SEED. KPTS.NRANPT. STEP 

3  FORMAT! 'CENTERING  BOX  WITH  ALL  RANDOM  XX.  SEED  *  '.D13.5./ 

*  i'  KPTS.NRANPT.STEP  ■  '.2I5.F10.A) 

CALL  OBO!XX.NX.KPTS.FF.XMIN.XMAX.XIHIN.XIMAX 

*  fXI.NIC.X.IOK) 

NFUNC=MAX0! KPTS.NRANPT) 

IFdOK.GT.O)  00  TO  900 
00  TO  1000 
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IRUN=4  ....  nONTINUF  WITH  XX  AND  FF  AS  LEFT  BY  PREVIOUS  RUN 

400  URITE(Af4)  KPTS f NLOOPS f STEP 

4  FORHAK 'OnONTINUING  BOX  WITH  SANE  XX' »/ 

»  »'  KPTS»NLOOPS»STEP  =  '»2I5tF10.A) 

NFIJNn=0 
GO  TO  900 

IRUN=5  ....  SET  INITIAL  XX  RANDOHLY  AROUND  LAST  XSTAR 

500  WRITE<*f6)  SEEDf KPTSiSTEP»XSTAR 

6  FORMAT( 'ORESTARTING  BOX  AROUND  XSTAR»  SEED  ='»Dl.s.5»/ 

*  »'  KPTSfSTEP  =' » I5fF10.<f.»/ 

*  XSTAR  ='»lP9F12.4f/»(t2X»lP9E12.4)) 

CALL  6B1 (XXf NX»KPTS»XSTARtFF»XHIN»XMAX»XIMIN»XIMAXf XI 

«  rNICflOK) 

NFUNr=KPTS 

IFCIOK.GT.O)  GO  TO  900 
GO  TO  1000 

NOW  CALL  BOX  WITH  XX  AND  FF  TO  FIND  OPTIMUH  XSTAR  AND  FSTAR 

900  CALL  TIMFS(nnATE»DTIHF»IVCPUl flTCPU) 

CALL  GOBOX<XX»NX»KPTS»NIC»XMIN»XMAXrXIHIN»XIHAX»FF»XI 

*  »XSTAR,FSTAR»MLOOPS»NFllNC) 

FSTAR=HAXMIN*FSTAR 

CALL  TIMES<nnATEf DTIME»IVnPU2» ITCPU) 

TIME=.00001.T*(IVCPU2-IVCPU1  ) 

IF(IPRINT.GT.O)  WRITE(Af5)  MLOOPS»NFUNC»TIHErFSTAR»XSTAR 

5  FORNAT( '0' » I5» '  LOOPS»'fIA»'  FUNCSr  AND'»F7.3»'  SECS  LATER  BOX' 

FINDS  FSTAR  AND  XSTAR  OF ' » IPEl A . 9 » / f ( 1P9E1 4 . A )  ) 

1000  RETURN 
END 
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SUBROUTINE  GOBOX(XXtNX>KPTS»Nir»XNINfXHAX»XJHINfXJHAX 
«  >PF»XI>XCfFSTARf NLOOPSrNFUNC) 

IMPLICIT  REAL*8  (A-Ht  0-Z) 

DIMENSION  XX(NXfKPTS)fXMIN(NX) fXMAX(NX) tXlMIN(NIC) iXIMAX(NIC) 
DIMENSION  FF<KPTS)»XI<NIC)»XC<WX) 

COMMON  /BOXCOM/  STEPtRELTOl.t  ABSTOL  »NTOL»NlCUTSf  N2CUTSf  NLOOPS 
»  » IPRINTrMAXMTN 

DATA  lOfll  /  Oft  / 

RIVEN  AN  INITIAL  COMPLEX  OF  KPT8  MX-VECTORS  IN  XXt  AND  KPTS 
FUNCTION  EVALUATIONS  IN  FFf  GOBOX  WILL  TRY  TO  FIND  THE  POINT 
WHICH  MAXIMIZES  FUNC  ON  THE  FEASIBLE  REGION.  THIS  POINT  IS 
RETURNED  IN  ARRAY  XCf  AND  ITS  VALUE  IN  FSTAR. 

FIND  HI  AND  LO  POINTS  IN  THE  COMPLEX 


JLO=l 
JHT  =  1 

DO  40  J=2fKPTS 

IF<FF(  J)  .LT.FFC  Jl.O)  >  JLO*J 
IF<FF<  J)  .OT.FF<.JHI))  JHI=J 
40  CONTINUE 
1101=0 

IF<IPRINT-2)  lOOfSOf^O 
50  F=MAXNIN*FF< JHI) 

WRITE  (Af2)  lOf  Ilf  JHI.Ff  (XXdf.lHDf  I«lfMX) 

60  TO  too 
60  DO  70  J»IfKPTS 

F=MAXHIN*FF< J) 

IF(J.NE.JHI)  URITE(6fl)  lOf I 1 f Jf F f ( XXt I f J) f I-l f NX) 

IF( J.EO.JHI)  WRlTE(6f2)  JOf 11 f Jf Ff (XXt I f J) f 1-1 f NX ) 

1  FORMATt'  LPfNFf  JfFfX='f3I3f  lPE12.4f  lXf7En.3f/ 

*  f (37XflP9Etl,3)) 

2  FORMATt'  *»LPfNFf.lfFfX»'’f3I3flPE12.4flXf9E11.3f/ 

*  f (37Xf 1P9E11 .3) ) 

70  CONTINUE 

MAKE  UP  TO  NLOOPS  ITERATIONSf  EACH  TIME  IMPROVING  THE  LOWEST  VALUE 

100  DO  500  ILOOP«lf NLOOPS 

LOCATE  NEXT  LOWEST  POINT 

N0RMLC»1 
IFUNC»NFUNC 
NEXTLO«JHI 
DO  140  .J»lfKPTS 

IFtJ.NE.JLO  .AND.  FFt  J) .  1.  T .FF tNEXTLO)  )  NEXTLO-J 
140  CONTINUE 

C 
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FIND  CFNTROin  AND  CHECK  ITS  FEASIBILITY. 

CALL  CENTER(XX.NX»KPTS» JLOtXC) 

CALL  FFASBl (XCtNX»XC>XHINfXNAX>XIHTNf XINAXfXIfNIC 
»  .NHOVESfO) 

IFtNMOVES.GT.O)  GO  TO  200 

FOR  FEASIBLE  CENTROIDf  LOOK  FOR  NEW  POINT  BY  MOVING  FROM  LO  POINT 
THRU  CENTER  BY  FACTOR  OF  STEP»  THEN  HOVE  TOWARDS  CENTER  IF  NECESSARY 
TO  GET  A  POINT  BOTH  FEASIBLE  AND  BETTER  THAN  NEXTLO  POINT. 

DO  ISO  1=1 » NX 

150  XX(I» JLO)=XX(I. JL0)+<1 .+STEP>*<XC(I )-XX(If JLO) > 

CAl  I..  MOVE(XX(  1 .  JLO)  f  NX»XC»NHOVES»NtCUTS»FF(JLO)  »FF(NEXTLO) 

«  iNFUNC*XMIN»XMAX>XIMIN»XIMAX>XI»NIC> 

IF<NM0VES.I.E.N1CUTS)  GO  TO  400 

IF  ABOVE  DIDN'T  WORKf  OR  IF  CENTER  NOT  FEASIBLEt  LOOK  FOR  NEW 
POINT  BY  MOVING  FROM  CENTER  TOWARDS  HI  POINT. 

200  DO  250  1=1. NX 

250  XX< I » JLO)=XC( I ) 

N0RMLC=-1 

CALL  MOVE  <  XX  < 1 . JLO ) . NX . XX  < 1 . JHI ) » NMOVES » N2CUTS 
»  fFFiJl.O)  tFF(NEXTl.n)  fNFUNC 

*  . XMIN.XMAX.XIMIN.XIMAX.XI.NIC) 

IF(NM0VES.l.E.N2CUTS)  GO  TO  400 

IF  THIS  DIDN'T  WORK.  TEST  ON  BOTH  SIDES  OF  HI  POINT  ALONG  LINE 
THRU  CENTER  FOR  FEASIBILITY  AND  IMPROVEMENT,  IF  BOTH  ARE  INFEAS¬ 
IBLE  nillT  WITH  INFEASIBLE  DIRECTION  AT  8001  IF  ONE  IS  BOTH 
FEASTBIE  AND  BETTER.  CONTINUE  WITH  IT»  OTHERWISE  QUIT  WITH  NO 
IMPROVEMENT  AT  700 

CAL  I.  FEASBl.(XX(  1 .  Jl.O)  .NX.XXIl .  JLO)  .XMIN.XMAX.XIMIN.XIMAX 
«  .XI .NIC. NMOVES. 0) 

IF<NMOVES.l E.O)  GO  TO  700 
DO  260  1=1. NX 

260  XX(I. JLO)=XX<I. JHI)+<XX(I.JHI)-XX(I. JLO) ) 

CALL  FEASBL(XX(1 .  JLO)  .NX.XXd.  Jl.O).XNIN.XNAX.XININ.XIHAX 
«  .XI. NIC. NMOVES. 0) 

IFINMOVES.GT.O)  GO  TO  800 
CALL  FUNC <  XX  < 1 . JLO ) » NX . FF ( JLO ) ) 

NFUNC=NFUNC+1 

IF(FF( JLO) .GT.FF(NEXTLO))  GO  TO  400 
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C  NOU  THAT  UE  HAVE  IMPROVED  THE  LOH  Xt  TEST  FOR  COMPLETION  AND 
C  LOOP  AROUND  AGAIN 
C 

400  MFUNC=NORNI.C*<NFIiNC-IFUNC) 

F*MAXMIN*FF(  Jl.O) 

IF(FF<  JLO)  .I.E.FF<  JHI)  >  60  TO  440 
IF(IPRINT.GE.2)  URITE<A>2)  HOOP . MFUNC t JLO » F 
»  »  (XXdf  JI.O)  fl^lfNX) 

JHI=JLO 
GO  TO  450 

440  IF(IPRINT.GE..T)  URITE(6»1)  IL OOP i MFUNC > JLO •  F 

«  >(XX(I>JLO)fIsl>NX> 

450  JLO=NEXTLO 

IT0L=IT0L+1 

ATOL*DAPS<FF( JHI)-FF<JLO> ) 

IF( ATOL.LE.ABSTOL)  GO  TO  480 
IF<ATOL.GT.RELTOL*DABS(FF(JHI) ) )  IT0L»0 
480  IF( ITOL.GE.NTOL)  GO  TO  AOO 

500  CONTINUE 


ML00PS=NL00PS+1 
GO  TO  900 
C 

600  NLOOPS=ILOOP 
GO  TO  900 
C 

700  MLOOPS—TLOOP 
GO  TO  900 
C 

800  MLOOPS=-NLOOPS-I 
C 

900  no  950  1*1 »NX 
950  XC<I)-XX<I» JHI) 
FSTAR»FF< JHI ) 

C 

RETURN 

END 


NO  CONVERGENCE 

CONVERGENCE 

UNABLE  TO  IMPROVE 

INFEASIBLE  DIRECTION 


n  n  rj  n  n  n 


SUBROUTINE  FEASBL ( X »NX» XC » XMIN t XHAX> XIMIN> XIMAX > X I »NIC 
»  fNHOUESfMAXHOU) 

IMPLICIT  REAL*8  <A-H»0-Z) 

DIMENSION  X(NX) f XC(NX)»XHIN(NX) »XMAX(NX) 

DIMENSION  XIMTN<NIC)  .XIMAX(NIC)  fXKNID 

COMMON  /BOXCOH/  STEP#  REl.TOL. » ABSTOL  f  NTOL » N1  CUTS » NPCUTS » Nl.OOPS 
*  f IPRINTf MAXMIN 

C 

C  TRIES  TO  MAKE  VECTOR  X  FEASIBl E  BY  (IF  NECESSARY)  MOVING  X 
C  TOWARDS  GIVEN  FEASIBLE  VECTOR  XC  UP  TO  MAXMOV  TIMES, 

C 

C  CHECK  AGAINST  EXPLICIT  CONSTRAINTS 
C 

NM0VES=0 

100  DO  200  I=1»NX 

IF(X(I)  .t.T.XMTN(I)  .OR.  X  ( I  > .  OT .  XMAX  ( I ) )  GO  TO  400 
200  CONTINUE 

CHECK  AGAINST  IMPLICIT  CONSTRAINTS 

IF<NIC.l.E.O)  RETURN 
CALL  TMPLCT(X»NX»XI»NIC) 
no  ,100  I  =  1»NIC 

IF<XI(I).LT.XIMIN<I)  .OR.  XI ( I ) . GT . XIMAX ( I ) )  60  TO  400 
300  CONTINUE 

RETURN 

INCREMENT  NMOVES  AND  IF  .LE,  MAXMOV r  MOVE  X  TOWARDS  CENTROID, 

400  NM0VES=NM0VES+1 

IF.<NMOVES.GT. MAXMOV)  RETURN 
DO  SOO  I=lfNX 
500  X<I)=0.5»(X<I)+XC<I)) 

GO  TO  100 
C 

END 
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SUBROUT  TNE  MOVE ( X  *  NX  *  XT » NHOUES  >  HAXHDV  t  E  t  EHINt  NEUNC 
»  »XHIN>XHAX»XIHIN>XIHAXfXI>NIC) 

IMPLICIT  REAL*8  <A-HfO-Z) 

DIMENSION  X(NX) iXT(NX)>XHIN(NX)>XMAX<NX}»XIMIN(NIC) 
DIMENSION  XIHAX(NIC>»XI(NIC> 

TRIES  TO  VERIEY  THAT  VECTOR  X  IS  BOTH  EEASIBI.E  AND  HAS  VALUE 
.GT.  EMINf  IE  NECESSARY  BY  MOVING  X  TOWARDS  THE  VECTOR  XT  UP 
TO  MAXMOV  TIMES. 

NM0VES*0 

100  CALL  EEASBLLX>NX»XTfXHIN»XMAX»XIMIN»XIMAXfXT»NIC 
»  t IMOVES  >  MAXMOV-NMOVES ) 

NMOVES=NMOVES+IMnVES 
lELNMOVES.GT. MAXMOV)  RETURN 
CALL  EUNCCX.NXfE) 

MEUNC*NEUNC+1 
IE<E.GT.EMIN)  RETURN 
DO  200  I=lfNX 
200  X<I)»0.5*<X(I)+XT<I)) 

NM0VES=NM0VES+1 

IE <NMOVES. IE. MAXMOV)  GO  TO  100 
END 


SUBROUTINE  CENTER < XX » NX tKPTS » NOTJ t XC ) 

IMPLICIT  REALBS  <A-H»0-Z) 

DIMENSION  XX<NX»KPTS)fXC(NX> 

COMPUTES  THE  NX-VECTOR  XC  AS  THE  CENTROID  OF  THE  KPTS  NX-VECTORS 
IN  XX  LESS  THE  ONE  INDEXED  BY  NOTJ. 

DIVKM1»1.0/<KPTS-1) 

DO  200  1=1 »NX 
XSUM=0. 

DO  100  J=1.KPTS 
100  XSUM=XSUM+XX<I»J) 

XC< I )=<XSUM-XX< I fNOTJ) )#DIVKM1 
200  CONTINUE 

RETURN 

END 


47 


n  n  o  n  n 


SUBROUTINE  6B0<XX»NX»KPTS,FFf XMINrXMAXf XIMIN»XIHAX 
«  >XI»NlC»Xf lOK) 

IHPLICIT  REAL*8  (A-H»0-Z) 

DIMENSION  XX(NX»KPTS) ,FF<KPTS) t XMIN(NX) »XMAX<NX) 

DIMENSION  XIMIN(NIC) »XIMAX<NIC>  »XI (NIC) »X(NX) 

COMMON  /BOXCOM/  STEP»RELT0L»ABST0LfNT0L»NlCUTS»N2CUTS»NL00PS 
*  » IPRINT  ^MAXMIN » SEED  f NRANPT » MAXRAN 

C 

C  CREATES  A  RANDOM  INITIAL  COMPLEX*  XX*  BY  CHOOSING  THE  BEST  KPTS 
POINTS  OUT  OF  A  COLLECTION  OF  MAX(KPTS*NRANPT)  RANDOM  VECTORS* 
EACH  UNIFORMLY  DISTRIBUTED  OVER  THE  FEASIBLE  REGION. 

FILL  XX  WITH  KPTS  FEASIBLE  RANDOM  VECTORS 

NRANX=0 

MXRANX-MAXRAN*MAXO (KPTS  *  NRANPT  > 

DO  300  J=1*KPTS 
100  DO  200  1=1 *NX 

200  XX ( I  *  J ) =XMIN ( I ) +RAN2 ( SEED ) % ( XMAX ( I > -XM IN ( I ) > 

NRANX=NRANX+1 

IF<NRANX.GT.MXRANX)  GO  TO  1100 

CALL  FEASBL(XX<1*J) *NX*XX(1* J) *XMIN*XMAX*XIMIN*XIMAX*XI 
»  *NIC*NM0VES*0) 

IF(NMOVES.GT.O)  GO  TO  100 
CALL  FUNC(XX(1* J) *NX*FF(J) > 

300  CONTINUE 

K1=KPTS+1 

IF (Kl.GT. NRANPT)  GO  TO  1000 
JMINF=1 

DO  500  J=2*KPTS 

500  IF(FF< J) .LT.FF( JMINF) )  JMINF=J 


C  CHOSE  (NRANPT-KPTS)  RANDOM  VECTORS»  ALWAYS  KEEPING  IN  XX 
C  THE  BEST  KPTS  VECTORS  LOOKED  AT  SO  FAR. 

C 

DO  900  K=K1»NRANPT 
600  DO  650  1=1 f NX 

650  X(I)=XMIN(I)+RAN2<SEED)*(XMAX(I)-XMIN(I) ) 

NRANX=NRANX+1 

IF(NRANX.GT.MXRANX)  GO  TO  1100 

CALL  FEASBL(XfNXfX»XMIN>XMAX>XIMlNfXIMAX>XIfNlC 
«  fNMOVESfO) 

IF<NMOVES.GT.O)  GO  TO  600 
CALL  FUNC(X»NX»F) 

IF(F.LE.FF( JMINF) )  GO  TO  900 
FF( JMINF)=F 
DO  700  I=1»NX 
700  XXdf  JMINF)=X(I) 

JMINF=1 

DO  800  J=2»KPTS 

800  IF<FF( J) .LT.FF( JMINF) )  JMINF=J 

900  CONTINUE 

C 

1000  I0K=1 
RETURN 
C 

1100  URITE(6.3)  NRANX 

3  FORMAT < '0**#)K»  IN  GBOf  WE  HAVE  EXCEEDED  THE  MAX  NUMBER' 
*  r'  OF  RANDOM  POINTS  ('fI6f'  )') 

I0K=0 

RETURN 

END 
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SUBROUTINE  GBl (XXtNXt KPTS»XO»FF»XMIN»XMAXrXIMINf XIMAX»XI 

*  »NIC»IOK) 

IMPLICIT  REAL»8  (A-H»0--Z> 

DIMENSION  XX(NXrKPTS) »XO(NX) »FF(KPTS>  »XMIN(NX) fXMAX(NX) 
DIMENSION  XIMIN(NIC) fXIMAX<NIC>  fXI(NIC) 

COMMON  /BOXCOM/  STEP»RELT0L»ABST0L»NT0L»NlCUTSfN2CUTSfNL00PS 

*  f IPRINTfMAXMINfSEEDfNRANPTfMAXRAN 

CREATES  A  RANDOM  INITIAL  COMPLEX »  XX »  AROUND  AND  INCLUDING 
A  GIVEN  FEASIBLE  VECTOR  XO. 

CHECK  XO  FOR  FEASIBILITY  AND  THEN  PUT  IT  IN  XX 

CALL  FEASBL(XO.NX,XO»XMIN»XMAX»XIMINfXIMAXfXI»NIC 

*  .NM0VES»0) 

IF(NMOVES.GT,0)  GO  TO  1000 
DO  50  1=1 f NX 

50  XXdf  1)=X0<I) 

CALL  FUNC(X0»NX.FF(1 )  ) 

C 

C  FILL  REST  OF  XX  WITH  RANDOM  VECTORS  WHICH  HAVE  BEEN  FORCED 
C  FEASIBLE  BY  MOVING  THEM  TOWARDS  XO  IF  NECESSARY, 

C 

NRANX=0 

MXRANX=MAXRAN*(KPTS-1 ) 

DO  400  J=2rKPTS 
100  DO  200  1*1, NX 

200  XX < I , J ) =XM IN ( I ) +RAN2 < SEED ) ♦ < XMAX ( I ) -XMIN ( I >  > 

NRANX=NRANX+1 

IF<NRANX,GT.MXRANX)  GO  TO  1100 

CALL  FEASBL<XX<1,J) ,NX,XO,XMIN,XMAX,XIMIN,XIMAX,XI ,NIC 

*  ,NM0VES,N2CUTS) 

IF(NM0VES,GT,N2CUTS)  60  TO  100 
CALL  FUNC(XX<1, J) ,NXtFF< J) ) 

400  CONTINUE 

C 

I0K  =  1 
RETURN 
C 

1100  WRITE(6,2)  NRANX 

2  FORMAT < 'O*#***  IN  GBl,  WE  HAVE  EXCEEDED  THE  MAX  NUMBER' 

»  OF  RANDOM  POINTS  (',16,'  )') 

I0K=0 

RETURN 

C 

1000  WRITE(6,1)  XO 

1  FORMAT < '0***»*  NON-FEASBIBLE  INITIAL  POINT  GIVEN  TO  GBl',/ 

»  ,'  XO  =' ,1P10E12.4> 

I0K=0 

RETURN 

END 
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SUBROUTINE  FUNC(X»N»F) 

IMPLICIT  REAL«8  (A-H»0-Z) 

DIMENSrON  X(N) 

COMMON  /BOXCOM/  STEP  f  REL  TOL  t  ABSTOI. » NTOL  t  N1  CUTS  t  NZCUTS  t  NLOOPS 
»  f IPRINTfNAXMIN 

DIMENSION  Y(16) 

DATA  Y/747e0.  f  28610.  t ?.1AS0. » 1 f  1A.170.  f  13770 .» 1 1 540.  f  9744.  tB26i 
S  . t7030. »6005. f 5147. »4427. »3820. f3307. »2872,/ 

RESO(A.B»nfY*T)  «  <Y  -  AtDEXP(B/(T-fC> )  )»«2 

C 

C  COMPUTES  F*  THE  VALUE  OF  THE  FUNCTION  WHEN  EVALUATED 
C  AT  THE  POINT  X  (AN  N-VECTOR). 

C 

F  =  0. 

DO  100  J-l»16 
T  =  50.+5.*(J-l) 

100  F  *  F  +  RESQ<X(l)fX<2)»X<3)»Y(J)»T) 

F  =  DSQRTCF) 

C 

F=MAXMIN»F 

RETURN 

END 


SUBROUTINE  IMPLCT(X.NX»XIf NIC) 

IMPLICIT  REAL*8  <A-H»0-Z) 

DIMENSION  X(NX) f XI(NIC) 

COMPUTES  XI »  THE  NIC-VECTOR  OF  IMPLICIT  CONSTRAINT 
FUNCTION  VALUES  AT  THE  NX-VECTOR  X. 

XI(l)  =  X<2)/<50.+X<3) ) 

C 

RETURN 

END 


51 


UNCLASSIFIED  /  ‘ 


SKUHTyja^HCATION  Of  THIS  PAGt  (When  Data  Knund)  ' 


1  '  I  I  REPORT  DOCUMENTATION  PAGE 

READ  INSTRUCTIONS 

BEFORE  COMPLETING  FORM 

'  T.  REPC^T^NUMBEIt  7.  GOVT  ACCESSION  NO. 

;es^.8ms.s  /  . 

3.  REOPIENrS  CATALOG  NUMBER 

4.  TITLE  (and  SublUte}  ' 

^  Practical  Aspects  of  Nonlinear  Optimization  ^  ^ 

..  5.  TYPE  OF  RtEOC-*  PERIOP  eOWERED 

1  Technical ^epMt.  .  1 

’’SrWiroBiiiNa  o»o.  Rcpeitr  number 

Technical  Report  576 

7.  AUTHOIKs) 

i 

’  1  Richard  B.  Holmes  amI  Jeffrey  Wi  Tolleson  t  ^  J‘ 

" -  — 

• 

8.  CONTRAO  OR  GRANT  NUMBERt» 

^  F19628.8(r-C^^y 

9.  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS 

Lincoln  Laboratory,  M.l.T. 

P.O.  Box  73 

Lexington,  MA  02173 

10.  PROGRAM  ELEMENT  PROJEO,  TASK 

AREA  A  WORK  UNIT  NUMBERS 

Program  Element  Nos.  63304A 
and  63398A 

1 1 .  CONTROUING  OFFICE  NAAilE  AND  ADDRESS 

Ballistic  Missile  Defense  Program  Office  / 

Department  of  the  Army  ■' 

5001  Eisenhower  Avenue  ^ - 

Alexandria,  V  A  22333 

■tr  REPORT  DAH 

^  19  Jun«M«ei  ' 

i3.  NUMBER  OF  PAGES  ,  . 

58  ^  1 

U.  MONITORING  AGENCY  NAME  A  ADDRESS  (ifdiffmM  /ram  CofumOing  Offict) 

Electronic  Systems  Division 

Hanscom  AFB 

Bedford,  MA  01731 

15.  SECURITY  aASS.|</ilu>  report)  '' 

Unclassified 

15a.  DECLASSIFICATION  DOWNGRADING 

SCHEDULE 

16.  OISTMM/TION  STATEMENT  (o/thu  Report) 

Approved  for  public  release;  distribution  unlimited. 

17.  DISTRIBUTION  STATEMENT  (<^  the  atutrart  entered  in  Block  20,  different  from  Report) 

18.  SUPPIEMENTARY  NOTES 

None 

19.  KEY  WORDS  (Continue  on  reverte  tide  if  necettary  and  tdentifr  hy  bloek  numher) 

nonlinear  programming  method  computer  code 

30.  ABSTRACT  (Corttinue  on  reverte  ttde  if  neeeuary  and  identify  by  block  number) 

A  general  purpose  nonlinear  programming  method  and  computer  code  are  presented.  The  method  is 
basically  heuristic  but  extremely  simple  and  reliable.  Interactive  operation  of  the  code  and  its  performance 
on  several  test  problems  is  described. 

00 


ra(M 

1  JAN  73 


1473  fomoNor  iNOVdsisoNOun 


_ UNCLASSinED _ 

MCUMTV  CLASSmCATON  07  THIS  7A0f  fWItm  Dam  EmtfWJ 


